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Abstract —This paper develops efficient algorithms for dis¬ 
tributed average consensus vrith quantized communication using 
the alternating direction method of multipliers (ADMM). We first 
study the effects of probabilistic and deterministic quantizations 
on a distributed ADMM algorithm. With probabilistic quanti¬ 
zation, this algorithm yields linear convergence to the desired 
average in the mean sense vrith a bounded variance. When 
deterministic quantization is employed, the distributed ADMM 
either converges to a consensus or cycles with a finite period 
after a finite-time iteration. In the cyclic case, local quantized 
variables have the same mean over one period and hence each 
node can also reach a consensus. We then obtain an npper bound 
on the consensus error which depends only on the quantization 
resolution and the average degree of the network. Finally, we 
propose a two-stage algorithm which combines both probabilistic 
and deterministic quantizations. Simulations show that the two- 
stage algorithm, without picking small algorithm parameter, has 
consensus errors that are typically less than one quantization 
resolution for all connected networks where agents’ data can be 
of arbitrary magnitudes. 

Index Terms —Quantized consensus, dither, probabilistic quan¬ 
tization, deterministic quantization, alternating direction method 
of multipliers, linear convergence. 

I. Introduction 

I N recent years there has been considerable interest in 
distributed average consensus where a group of agents aim 
to reach a consensus on the average of their measurements 
El-Eol. This is largely motivated by numerous applications 
in control, signal processing, and computer science. For ex¬ 
ample, the distributed averaging is a fundamental problem in 
ad hoc network applications, such as distributed agreement 
and synchronization a, distributed coordination of mobile 
autonomous agents a, and distributed data fusion in sensor 
networks a. It has also found applications in load balancing 
for parallel computers a- 

We consider in this paper distributed averaging algorithms 
where nodes only exchange information with their immediate 
neighbors. These algorithms are extremely attractive for large 
scale networks characterized by the lack of centralized access 
to information. They are also energy efficient and enhance 
the survivability of the networks, compared with fusion center 
based processing. However, a number of factors such as lim¬ 
ited bandwidth, sensor battery power, and computing resources 

This work was supported by National Science Foundation under Award 
CCF1218289, by Army Reseai'ch Office under Award W91 lNF-i2-l-0383, 
and Air Force Office of Scientific Research under Awai'd FA9550-10-1-0458. 
The material of this paper was presented in part at IEEE GlobalSIP 2015 [fl. 

S. Zhu and B. Chen are with the Department of Electrical Engineering and 
Computer Science, Syracuse University, Syracuse, NY 13244 USA (e-mail: 
szhu05 @ syr.edu; bichen @ syr.edu). 


place tight constraints on the rate and form of information 
exchange amongst neighboring nodes, resulting in quantized 
consensus problems E, E- This paper is specifically devoted 
to developing efficient algorithms for quantized consensus in 
connected networks with static topologies. 

A. Related work 

There are three widely used methods for solving distributed 
averaging problems. A classical approach is to update the 
state of each node with a weighted average of values from 
neighboring nodes Ei-ini. The matrix, consisting of the 
weights associated with the edges, is chosen to be doubly 
stochastic to ensure convergence to the average. Another 
method is a gossip based algorithm, initially introduced in ||2T]| 
for consensus problems and further studied in fS), ifT^ . ifTSl . 
among others. The third approach is to employ the ADMM 
which is an iterative algorithm for solving convex problems 
and has received much attention recently (see Il22l and ref¬ 
erences therein). The idea is to formulate the data average 
as the solution to a least-squares problem and manipulate the 
ADMM updates to derive a distributed algorithm M-M- 

In the most ideal case where agents are able to send and 
receive real values with infinite precision, the three methods 
all lead to the desired consensus at the average. When quan¬ 
tization is imposed, however, these methods do not directly 
apply. A well studied approach for quantized consensus is to 
use dithered quantizers which add noises to agents’ variables 
before quantization ll^ . By imposing certain conditions, 
the quantization error sequence becomes independent and 
identically distributed (i.i.d.) and is also independent of the 
input sequence. The classical approach and the gossip based 
algorithm then yield the almost sure consensus at a common 
but random quantization level with the expectation of the 
consensus value equal to the desired average ini, m, Eoi. 
To the best of our knowledge, there have been no existing 
results on the ADMM based method for quantized consensus. 
Nevertheless, since the quantization error of dithered quantizer 
is zero-mean and has a bounded variance, we can immediately 
extend the results in m, ca to quantized consensus (see 
Section lTVl i. That is, the ADMM based method using dithered 
quantization leads to the consensus at the data average in the 
mean sense whose variance converges to a finite value. 

Meanwhile, studies on distributed average consensus with 
deterministic quantizers have been scarcely reported. Deter¬ 
ministic quantization makes the problem much harder to deal 
with as the error terms caused by quantization no longer pos¬ 
sess tractable statistical characteristics El, El- The authors 
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in in show that the classical approach, where a quantization 
rule that rounds the values down is adopted, converges to a 
consensus with an error from the average depending on the 
quantization resolution, the number of agents, the agents’ data 
and the updated weights of each agent. A recent result of m 
indicates that this approach, with appropriate choices of the 
weights, reaches a quantized consensus close to the average 
in finite time or leads all agents’ variables to cycle in a small 
neighborhood around the average; in the latter case, however, 
the consensus is not guranteed. The gossip based algorithms 
in II20I and IH have similar results to those of the classical 
approach. The ADMM based algorithms for deterministically 
quantized consensus, however, have not yet been explored. 

B. Our contributions 

One shall note that the consensus error for deterministically 
quantized consensus in IfTTlI . 1201 is much undesired when the 
number of agents or the range of agents’ data becomes very 
large. Unfortunately, this is typically the case in large scale 
networks or big data settings. The ADMM has been known to 
be an efficient algorithm for large scale optimizations and used 
in various applications such as regression and classification 
im. Moreover, l24ll - l^ validate the fast convergence of 
the ADMM and ifTsll . ifTbl demonstrate the resilience of 
the ADMM to noise, link failures, etc. We therefore expect 
ADMM based methods to work well for quantized consensus 
problems, with regards to both the consensus error and the 
convergence time. 

We first study the effect of probabilistic quantization fUX . 
which is equivalent to a dithering method as shown by ini 
Lemma 2], on the ADMM based method. Utilizing the first and 
second order moments of the probabilistic quantizer output, 
we establish the convergence to the average in the mean 
sense based on existing convergence results of the ADMM. 
Furthermore, recent work of l28l enables us to immediately 
characterize the convergence rate of the distributed ADMM 
with probabilistic quantization. 

The main contribution of this paper is to design and analyze 
an ADMM based approach using deterministic quantization. 
We establish that a distributed deterministically quantized 
ADMM algorithm either converges to a consensus or cycles 
around the average after a finite-time iteration as long as a 
mild initialization condition is satisfied. We also show that the 
cyclic period is finite and that the quantized variable at each 
node has the same mean over one period. Thus, a consensus 
can be reached within finite iterations for both convergent and 
cyclic cases. We then derive an upper bound that only depends 
on the quantization resolution and the average degree of the 
undirected graph (two times the ratio of the number of edges 
to the number of nodes). This is much preferred for large scale 
networks as it does not rely on the number of agents or the 
agent’s data. 

While numerical examples show that the deterministically 
quantized ADMM converges in most cases, we notice that 
it may reach different consensus values with different initial 
variable values. It is well known that a good starting point 
usually helps in such settings. This inspires our approach 


for quantized consensus which first uses the probabilistic 
method to obtain a good starting point and then employs the 
deterministic algorithm. Simulations show that this two-stage 
approach tends to converge and also performs best among all 
existing methods using deterministic quantization in terms of 
the consensus error. 

C. Paper organization 

The rest of this paper is organized as follows. Section 
HJ reviews the application of the ADMM to the distributed 
averaging problem without quantization, which leads to a 
distributed ADMM algorithm. We then develop several con¬ 
vergence results of this algorithm; they will be used later to 
establish our main results. Section |III] defines probabilistic 
and deterministic quantization schemes. Their effects on the 
distributed ADMM are studied respectively in Sections HVl and 
[Vl Section [Vl] describes the proposed algorithm for quantized 
consensus which combines the two quantized ADMM meth¬ 
ods, followed by simulation results in Section IVIII Section 
IVIIII concludes the paper. 

D. Notations 

Denote by ||£c||2 the Euclidean norm of a vector x and 
(a;, y) the inner product of two vectors x and y. Given a 
semidefinite matrix G with proper dimensions, the G-norm of 
X is IIa:IIG = Vx^Gx. Also denote crniax(Ai’) as the largest 
singular value of a square matrix M and amin{M) as the 
smallest nonzero singular value of M. 

We use two definitions of rate of convergence for an iterative 
algorithm. A sequence a:^, where the superscript k stands for 
time index, is said to converge Q-linearly to a point x* if there 
exists a number v € (0,1) such that limfe_,.oo k = u 

||a5 X II 

with II • II being a vector norm. A sequence is said to 
converge R-linearly to y* if for all k, ||y^ —y*|| < ||a;^ —a:*|| 
where a;^ converges Q-linearly to x*. 

II. Distributed Average Consensus by the ADMM 

This section introduces the consensus ADMM (CADMM) 
for average consensus without quantization. This ideal case 
provides a good understanding of how the ADMM works for 
distributed average consensus. We start with the setting of the 
distributed averaging problem. 

A. Problem setting 

Consider a connected network of N agents which are 
bidirectionally connected by E edges (and thus 2E arcs). 
We describe this network as a symmetric directed graph 
= {V,w4} or an undirected graph (/„ = {V,f}, where 
V is the set of vertices with cardinality |V| = N, ^ is the set 
of arcs with |yl| = 2E and S is the set of edges with \E\ = E. 
Assume that the topology of the network is fixed throughout 
this paper. Let S R be the local data only available at node 
i, i = 1, 2, • • • ,N, and r € R^ the vector concatenating all 
ri. The goal of distributed average consensus is to compute 
the data average 

1 ^ 

^avg = ^ ri (1) 

i=l 
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by local data exchanges among neighboring nodes. 


B. Application of the ADMM to distributed average consen¬ 
sus: CADMM 

The ADMM applies in general to the convex optimization 
problem in the form of 

minimize (yi) + 32 (y 2 ) 

yi,y2 ( 2 ) 

subject to Cifji C 2 y 2 = c, 


where yi and 1/2 are optimization variables, gi and 52 are 
convex functions, and Ciyi -\-C 2 y 2 = c is a linear constraint 
on yi and y 2 - The ADMM solves a sequence of subproblems 
involving gi and 52 one at a time and iterate to converge 
when, e.g., gi and 52 are proper closed convex functions and 
the Lagrangian of (|2]i has a saddle point ll2^ . 

To apply the ADMM, we first formulate ([T]i as a convex 
optimization problem 

Xavg = argmin^-(i - (3) 


that is, the data average is the solution to a least-squares 
minimization problem. We continue to reformulate (O in the 
form of (in as 


minimize 
subject to 



i=l 


Xi — Zij^Xj — G .A, 


(4) 


where Xi is the local copy of the common optimization 
variable x at node i and Zij is an auxiliary variable imposing 
the consensus constraint on neighboring nodes i and j. We 
emphasize that throughout the entire paper, Vi represents the 
local data, i.e., the observation at the ith agent, while Xi 
is referred to as the local variable. Since the network is 
connected, this constraint ensures the consensus to be achieved 
over the entire network, i.e., Xi = Xj,\/{i,j) G A, which in 
turn guarantees the solution to (|4]i is the data average a;avg- 
Further define x G as a vector concatenating all Xi, 
z G as a vector concatenating all Zij, and 

fix) = ^\\x-r\\l. (5) 

Then (HJi can be written in a matrix form as 
minimize f{x) + g{z) 

( 6 ) 

subject to Ax -|- Bz = 0, 

where g{z) = 0 , and 0 is a column vector with proper 
dimensions and all entries being 0. Here B = [—I 2 E', —I 2 E] 
with I 2 E being a 2E x 2E identity matrix and A = [Ai \ A 2 ] 
with Ai, A2 G If {i,j) G A and Zij is the qth entry 

of z, then the (g,t)th entry of Ai and the (q,j)th entry of 
A 2 are 1 ; otherwise the corresponding entries are 0 . 

We are now ready to apply the ADMM to solve the 
consensus problem. The augmented Lagrangian of (| 6 l) is 

Lp{x,z,X) = f{x) -f {\,Ax-GBz) -G ^\\Ax + Bz\\l, 

0) 


where A = [13]'f] with (3,^ G is the Lagrange multiplier 
and p is a positive algorithm parameter. At iteration fc +1, the 
ADMM first obtains x^~^^ by minimizing Lp{x, z^, A^'), then 
calculates 2 :^'+^ by minimizing Lp{x^~^^, z, \^) and finally 
updates A^+^ using and 2 ^+^. The updates are 

a;-update : V/(a;'=+^) + A'^A'^ 

+ pA'^iAx^+^ + Bz^) = 0 , 

2 -update : B^+ pB'^{Ax^+^ + Bz^+^) = 0, 
A-update : A'^+^ - - p{Ax^+^ + Bz^+^) = 0, 

where = x^^^ — r is the gradient of / at 

A nice property of the ADMM, known as global conver¬ 
gence, states that the sequence {x^,z^ , A^) generated by ® 
has a single limit point [x*, z* ,\*) which is a primal-dual 
solution to (|7]i- Proofs can be found in ||22|, ll24l . ll26l . Noting 
that our objective function f{x) given in (|5]l is strongly convex 
in X, we obtain x* = liavg as the unique primal solution 
where 1 denotes the W-dimensional column vector with all 
entries being 1. To summarize, we have 

Lemma 1 (Global convergence of the ADMM Il22\l . Il24\l . 
I[26\l): For any initial values x° G 2 ° G and A° G 
the updates in ([ 8 ]l yield that as /c —>■ 00 , 

x^ ->■ X*, z^ 2 *, and A'^ ^ A*, 


where (a;*, 2 *, A*) is a primal-dual solution to (|7]) and x* = 
ItCavg is unique for the distributed average consensus problem 

(Ell. 

While (Ull provides an efficient centralized algorithm to 
solve Ell, it is not clear whether ([Sll can be carried out in 
a distributed manner, i.e., data exchanges only occur within 
neighboring nodes. Interestingly, Lemma [T1 states that conver¬ 
gence for the ADMM is guaranteed regardless of initial values 
a;°, 2 ° and A°; there indeed exist initial values that decentral¬ 
ize (©. Define AL|_ = A^ + A|^ and AT_ = A^ — Aj which 
are respectively the unoriented and oriented incidence matrices 
with respect to the directed graph Qd- Initialize = — 7 ° and 
2 ° = ^M^x°. As shown in ll^ . the updates in ((81) lead to 




1 

1 + 2 p|A/i| 



a. 


fe+i 


= a. 


■p IM|< 


fe-i-i 




(9) 


at node i, where A/i denotes the set of neighbors of node i 
and is the Ah entry of = M-[3^ G Obviously, (O 
is fully decentralized as the updates of x^~^^ and only 

rely on local and neighboring information. Therefore (O can 
be used for distributed average consensus. We refer to (lUl as 
the CADMM for distributed average consensus. 

If we further initialize /3° in the column space of Alf 
(e.g., /3° = 0 ), then [3^ lies in the column space of Mf and 
converges to a unique f3*. We will use this result immediately 
but postpone its proof to Lemma |4] Note that this implies 
in (|9l) converges uniquely to a* = AT_/3*. We also notice an 
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interesting relation between a. and /3 even though Af_ is 
rank deficient^ 

Lemma 2: Given a connected network, if ^ lies in the 
column space of then a and (3 are one-to-one corre¬ 
spondence, i.e., for a. = M-(3 and a' = M-f3' where /3 
and f3' are in the column space of M'E, a. = a' if and only 
if 13 = 13'. 

Proof: That 13 = (3' implying a = a' is straightforward. 
Consider a. = M-(3 and write (3 = Mfb for some b S 
ot',f3' and b' are similarly defined. Then we have 

||a-a'||2 = ||M„M^(6-6')ll2 

>an,in(M„)||Ml(6-6')||2 
= a„,in(M„)||/3-/3'||2, 

where is the smallest nonzero singular value of 

Af_, whose existence is guaranteed for a connected graph 
ll29l . We therefore have (3 = (3' if oc = a!. ■ 

It is therefore meaningful to define an initialization condi¬ 
tion for the CADMM. A similar global convergence property 
for the CADMM is given in Lemma [3] 

Initialization condition for the CADMM: can be 

any vector in and lies in the column space of 
M^Mf. 

Lemma 3 (Global convergence of the CADMM): For any 

and q:° satisfying the initialization condition, the CADMM 
leads to 

—>■ X* and ^ a* as k ^ oo, 

where x* = ISavg and a.* = r — IXavg which lies in the 
column space of are both unique. 

Proof: Global convergence follows from Lemmas [T] and |2] 
together with the fact that (3^ converges to a unique /3* which 
lies in the column space of Mf. 

Now taking /c c» in (|9]l and using the fact that X* = Xavg 
for i = 1, 2, • • • , W, we have 

Ct j — Tj atavg ■ 

■ 

Throughout the rest of this paper, we assume that the 
CADMM, wherever used, is initialized to satisfy the initial¬ 
ization condition. 

C. Linear convergence of the CADMM 

We investigate two properties of the CADMM; the first 
property is built on global convergence while the second 
considers the rate of convergence. 

Define = iAT+Mj and X_ = which are 

respectively the signless and signed Laplacian matrices with 
respect to Qu ■ Let W € ^ ^ be the degree matrix related to 

the underlying network, i.e., a diagonal matrix with its {i, i)th 
entry being the degree of node i and other entries being 0. 
Then W = i(L+ + L_) and Lemma 12] is an immediate result 

^As defined in Section ITFcl M^Mf = 2i_ where L_ is the signed 
Laplacian matrix of the connected undirected graph and always has 0 as its 
eigenvalue. See ED 


from the property of 1 ^ . We rewrite (| 9 ]) in the matrix 
form as 


a;fc+i = (Jjv + 2 pW)-\pL+x’^ - + r), 

^k+i +pL_x'^+^, 


or equivalently. 




Ds'' 


( 10 ) 


with 


and 


s'' = 


D = 


pDi)L^ 

P^L-DqL^ 

On 


X 

a.^ 


-Do 

In — pL^Do 

On 


Do 

pL^Do 

In 


( 11 ) 


where On denotes the N x N matrix with all entries being 
0, Do = {In + 2 pTV)“^, = r, and hence, = r. From 

Cl, we have 


It is thus interesting to investigate how £)'' behaves ask ^ oo. 
From (fTTT i. a logical approach is to study through the 
structures of L_,X+ and VF; fortunately, the global conver¬ 
gence property of the CADMM provides a simple argument 
to obtain a rough estimate of which, nevertheless, is 

good enough for our purpose in establishing the main results. 
Note that we also have D* = D°° and s* = [x*\a.*-,r*] = 
\x°°] a.°°; r°°] = s°° as our optima due to global convergence 
of the CADMM. Our result about D* is given below. 

Theorem 1 : Consider D defined in (fTTT i. Then 



Dn 

Di2 

Di 3 


On 

ail^ 

2-11^ 1 

D* = 

D21 

D22 

D23 

= 

On 

a 2 l'^ 

In - 


D31 

D32 

D33 


On 

On 

In 


for fixed ai, 02 G 

Proof: By Lemma[ 3 ] we have for any that satisfies the 
initialization condition. 



■ ■ 


1 

* 

_1 


l^avg 

= 

^00 

— 

1 

* * 

1_ 

— 

T* l^Cavg 

r 


Recall that s°° = If we fix ofi and r°, global 

convergence implies that s°° = s* regardless of the initial 
value a;°. Thus Du = O^r, i = 1 , 2 , 3 . Similarly, fixing x^ 
and r°, we must have Di2cf‘ = D22cx^ = 0 . Since a'' is 
initialized in the column space of M-Mf = 2 L_ where 
L_ is the signed Laplacian matrix of a connected undirected 
graph, Di2 and D22 must be respectively the products of 
some vectors ai and 02 in multiplying 1^, such that 
D12L- = D22L- = 0 . Knowing the form of Dji and 
Dj2, j = 1,2, we see that x°° and a.°° only depend on 
= r. Together with the facts that x^ = x* has each entry 
of itself reaching the data average that 

Q,oo _ j, _ i2;avg for any r, we validate £>13 and D23 as 
given in the theorem. The remaining blocks, JD32 and D33, 
follow directly from the matrix multiplication. ■ 
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Given global convergence, we now turn our attention to the 
rate of convergence of the CADMM. Recent work of ||25]| . 
12^ has established the linear convergence of the ADMM. 
Unfortunately, their results do not apply to the CADMM as 
their conditions are not satisfied here. In ll25l . the step size of 
the dual variable update, i.e., p in the A-update of need 
be sufficiently small while our CADMM has a fixed step size 
p that can be any positive number (see Remark |3] for further 
discussion on the choice of p). The linear convergence in ll2^ 
is established provided that either g{z) is strongly convex or B 
is full row-rank in (01). In our formulation, however, g{z) = 0 
is not strongly convex and B = [—I 2 E]—I 2 e\ is row-rank 
deficient. Nevertheless, we first give Lemma |4] with regards 
to the convergence rate of a vector concatenating 2 ; and /3. 
A more general result can be found in ll^ Theorem 1]. Our 
proof is similar to that of but simpler. 

Lemma 4 ( 0S] Theorem 1 ]): Consider the ADMM iteration 
(Is)) that solves (jb]). Define 


2 

and G = 

pl2E 

f32E 

1 T 

_ /3. 


O 2 B 

-pl2E_ 


u = 


where f3 is the dual variable. If we initialize z^ = 

(3^ = — 7 ° where 7 is the other dual variable and 
is in the column space of M'£, then for k = 0,1,---, 
z’^ = /3^ lies in the column space of 

and {x^,z^,f}^) converges uniquely to {x*,z*,j3*) with 
X* = Ixavg, = IjM’Jliavg and /3* being a vector in the 
column space of AT_. Furthermore, = [z^-,(3^] converges 
Q-linearly to its optimal u* = [z*;f3*] with respect to the 
G-norm 


- u* 


\h< 


1 


l + S' 


lilt'" - u* 


Ig’ 


( 12 ) 


where 
(5 = min 


f _ 

1 2 a 2 _(M+) ’ 


o’max(AT+) denotes the spectral norm or the largest singular 
value of Af+, and denotes the smallest positive 

singular value of 

Proof: See Appendix. ■ 

With this lemma, we can now establish the linear conver¬ 
gence rate of the CADMM . 

Theorem 2 (Linear convergence of the CADMM): Consider 
the matrix form of the CADMM in (fTOl i. If = [x^; of; r°] 
satisfy the initialization condition, then 


^ (1 + 

where S and are defined in Lemma |4] Therefore, is 
R-linearly convergent to s*. 

Proof: Notice that the initializations in Lemma |4] decen¬ 
tralize the ADMM iteration (| 8 ]) into the CADMM. Thus x^ is 
the same in the ADMM iteration (| 8 ]) and the CADMM iteration 
(|9]) while of = Then (02|) implies 


|a;fe+i _a ;*||2 < \\u'^-u* 


IG- 


We also have 

||a'=+i-a*||2 = ||M_(^^-+i-r)l|2 

<a„,ax(M_)||/3'=+i-n|2 

< yp<Tinax(M_)||M''+^ - U*\\g (13) 

< - U*\\g, 

where the last two inequalities are from the definitions of u 
and G, and (fT 2 l) . respectively. Thus, 

ll^fc+i _ s*||2 < ||a;^'+i _ -f ||a^-+i - a*||2 
- 11 “' - 


III. Quantized Consensus 

To model the effect of quantized communications, we 
assume that each agent can store and compute real values 
with infinite precision; however, an agent can only transmit 
quantized data through the channel which are received by 
its neighbors without any error. The quantization operation 
is defined as follows. Let A > 0 be a given quantization 
resolution and define the quantization lattice in R by 

A = {tA :teZ}. 


A quantizer is a function Q : R A that maps a real value 
to some point in A. Among all quantizers we consider the 
following two for distributed average consensus; 

1) Probabilistic quantizer Qp defined as follows: for y £ 
[fA,(f + l)A), 


Qp{y) — 


fA, with probability t + 1 — 

(f -f 1)A, with probability ^ — t. 

2) Rounding quantizer Qd which projects ?/ £ R to its 
nearest point in A: 


1 


Qdiy) =tA, if {t--]A<y<{t+-]A. (15) 


1 


We point out that probabilistic quantization is equivalent to 
a dithered quantization method (see im Lemma 2]) while 
rounding quantization is one of the deterministic quantization 
schemes. Throughout the rest of this paper, we use Q{y) (or 
y[Q] for ease of presentation) to denote the quantized value of 
y £ R regardless of its quantization scheme; we use Qp{y) 
(or y[Q^]) and Qd{y) (or y;Q^]) when it is necessary to specify 
the quantization scheme. Quantizing a vector means quantizing 
each of its entries. Define eg = t/[Q] — y as the quantization 
error. It is clear that 

|eQj,| < A and |eQ^| < ^A, for any y £ R. (16) 

As seen from Section El the CADMM has the advantage 
of global and linear convergence for solving the average 
consensus problem as long as the initialization condition is 
met. The authors in na, m have also shown the good 
behavior of the ADMM in distributed settings when noise 
or random link failures are imposed. The rest of this paper 
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is devoted to investigating the effects of the two quantization 
schemes defined in (fT4l i and (fTSl l on the performance of the 
CADMM. We remark that the results of probabilistic and 
rounding quantizations hold respectively for other dithered and 
deterministic cases, which will be elaborated in Sections HVl 
andlV] 


IV. Probabilistic Quantization 


For ease of presentation, we only study the probabilistic 
quantization defined in (fl4li . The results can be easily extended 
to any other dithered quantization as the only information 
used is the first and second order moments of the probabilistic 
quantizer output which are stated in the following lemma. See 
(Ell for a proof. 

Lemma 5 ( y l27] Lemma 2]): For every ?/ £ R, it holds that 


E[Qp( 2 /)] 


y and E 


{y-Qp{y)f 


The iteration in (|9]l now takes the form of 



~ 1 + 2^1^ I ’ 

= af + pi- Y. 4+0 . 

\ ieVi / 

(17) 

Notice that is also quantized at its own node for the 
{k + l)th update; the reason will be given in Remark |5] 
As illustrated in ifTsl . iteration (flTl) can be interpreted as a 
stochastic gradient update. Viewed from this point, the quanti¬ 
zation error causes x\ to fluctuate around the quantization-free 
updates (|9]l. Our convergence claims are given in Theorem [3 


Theorem 3: Let and satisfy the initialization condi¬ 
tion. The probabilistically quantized CADMM (PQ-CADMM) 
iteration dnii generates x^,i = 1, 2, • • • ,N, which converges 
linearly to the data average Xavg in the mean sense as fc — c». 
In addition, the variance of x^ converges to a finite value 
which depends on A and the network topology. 

Proof: Taking expectation of both sides of ( fT7b . we have 

= i+lp\K\ + p E ®[4qp]] 


We notice that the convergence of E[a;^] — 


vg does 

not indicate that reaches a consensus when k ^ oo. 
Nevertheless, a simple method fixes this problem. The idea 
is to calculate the running average 4 = > 1 at 

each node i. One can use similar steps in the proof of nsi 
Proposition 3] to show that x!^ has diminishing variance. By 
Chebyshev’s inequality, we then get the following corollary. 


Corollary 1: Let x^ = \ J2i=i • 
i, we have 


lim if = Xs: 

k—¥C!0 


for k > 1. For each node 


= 1 . 


V. Deterministic Quantization 
Deterministic quantization is usually much harder to handle 
as the quantization error is not stochastic. Unlike probabilistic 
quantization, the accumulated error term is very likely to blow 
up; there have been a few methods proposed to counter such 
difficulties (see in, ESI, a), yet the resulting algorithms 
either do not guarantee a consensus or reach a consensus 
with an error from the desired average that depends on the 
number of agents, the quantization resolution, and the agents’ 
data. Our approach will establish a finite upper bound on 
the accumulated error term and then use the property and 
the initialization condition of local Lagrangian multipliers to 
deduce the consensus reaching result. 

Let the local data x^ be also quantized for the (fc + l)th 
update at node i. The updates become 

= i + 2p|Ar,| + p E 4q.] - ’ 

- E ■ 

\ ieVi / 

(19) 

Rewrite 4[Q<il = 4 + e-g, with £ [-A/2, A/2) 

according to (iTsT l. Then the a^-update, i = 1,2, ••• ,A, is 
equivalent to 



-E[af] + r,j, 

E[4+1] =E[a/]-t-p(|M|E[4+i,]- ^ 

V j&ATi / 

(18) 

Noting that Lemma |5] implies E[a;^jQ^j] = E[a;/] and 

E[4[q ]] = E[4], we see that (fTsT i takes exactly the same 
iterations in the mean sense as the CADMM. By initializing 
cx^ in the column space of L-, E[q:°] = (Xq satisfies the 
initialization condition. The linear convergence of E\x\] to 
ccavg is thus ensured due to Theorem |2] 

Since Lemma |5] also indicates the bounded variance of 
quantization error, the second claim follows directly from CSl 
Proposition 3]. ■ 


or written in the matrix form, 

=(x^ + pL_x’^+^ + pL.e'^+J-, ( 20 ) 

where denotes the vector concatenating all Recalling 

the ideal CADMM update ( flOl ), we have the matrix form of 
(fTOl l as 

s'^+i = j:)(s'= + 4) + 4 ( 21 ) 

where = [eQ^;0;0] and = [Q] pL-e^^]Q\. It is 

important to note that is deterministic and hence the 
update (ISTT i is deterministic. Our main results are stated in 
the following theorem. 

Theorem 4: Consider the deterministically quantized 
CADMM (DQ-CADMM) iteration ( fT^ . Let x^ and q:° satisfy 
the initialization condition for the CADMM. Then there exists 
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a finite time iteration fco > 1 such that for k > kg all the that the initialization and /3^_j meet the condition of 

quantized variable values Lemma 01 we thus have 


either converge to the same quantization value: 


k k ^ 

^l[Qd] = ■■ ■ = 


or cycle around the average Xavg with a finite period T > 
2’ i-e-> , 7 = 1, 2, • • • , iV, and 


T T 

1 V L I / 1 V 

=*** = Tf;^ 


z=i 


rj- ^N[Qd] — ^[Qd] ■ (22) 

Z =1 


For both convergent and cyclic cases, we have the following 


error bound for G { 


Qd c \-^[QdY-^[Qd\ 




I 2E 

\XQd-X^^g\ < [ 9 


(23) 


where the upper bound is tight if the DQ-CADMM converges. 

Proof: We prove that the DQ-CADMM either converges 
or cycles after a finite-time iteration and then use this fact to 
derive the error bounds. 

We see from (l20l i that must lie in the column space of 
if is initialized in the column space of L-. Following 
(ED, we have 

= D + Dsl-^ + 


= D 






^k—i 


k-1 

E^ 

7=0 




(24) 


The first term is simply the ideal CADMM update which 
converges to a finite value. We will show that the accumulated 
error term Yl!i=i + Yl'jZo is bounded and 

hence that is bounded. Notice that is the ith 

update of the CADMM with the initial value Let 

be the vector that concatenates the primal 
and dual variables in the ADMM iteration with initial 
values and /3°_j = 0 corresponding to 

— [®Qd j ^]' With G defined in Lemma 0] we obtain 


Mi. 


\l=P 



k—i 

Qd 


2 

2 


< JpaLx(M+)||e^ 7 ||^ 

< ^piVAVLx(M+), 


where the last inequality is from ( [Tbb . Since Theorem [T] 
indicates the form of D*, we get D*s^~^ = 0, i.e., xl_- = 0 
and a*_. = 0. Therefore, = 0 from 

Lemma Eland the fact that zl_- = Noting also 


< + ]J ^CTmax(AT_)^ W'^k-i ~ “fc-illc 

+ (t/rri) 

X 

max (M+)v^, (25) 

where (a) is from Theorem |2] and (b) is due to Lemma 0] 
together with the fact that ul_^ = 0. Similarly, we have for 
J>1, 


< 1 


1 + .5 


c(M_) 


X -Act 

max 



(26) 


and when j = 0, 

< \^a^UM-)p^. (27) 


Therefore, 


i=l 7=0 2 

k k—1 

i=l j=0 

k 

< Il«^i 2 + E 

< 2^ania^{M^)pVN + (^1 + y Y^-^Cri„ax(M_)^ 

X ^Aa/^ (crmax(M+) -|- 

^ max {m^))J 2 (\f^s 

(28) 

where (a) is from (|25]) - (|27]) . Then (EHI) must be finite for k = 
1,2, • • • , as (5 > 0, and thus is bounded. An important fact 
from (I 2 D is that the update of and hence is fully 
determined by due to the deterministic quantization 

and the CADMM update. Recalling that ||sj ||2 = ||eQ ^||2 < 
yVN and that -b = [x^Qj^\of]r\ with each entry of 
X[Qd\ being a multiple of A, each entry of a being a multiple 
of pA, and r being fixed, we conclude that there are only finite 
possible states of -b s^. Therefore, is either convergent 
or cyclic with a finite period T >2 after a finite-time iteration. 

We next consider error bounds for the consensus value. The 
consensus error may be studied directly by calculating the 
accumulated error term in (l24l i. However, the bound in (l28T l 
is quite loose in general since it results from the worst case. 
We alternatively derive the error bounds in the respective case 
using the fact that the DQ-CADMM either converges or cycles. 


























Convergent case: The convergence of the DQ-CADMM 
implies that for k > ko, and hence 


Summing both sides of (O over one period and dividing the 
sum by T, we have 


0 = 

Since L_ is the Laplacian matrix of a connected graph Q^, we 

must have that xftl reaches a consensus. Now let xfr^ i G A 
IQa] IWd] 

denote the convergent quantized value. Then 

for i = 1,2,-•• ,N, and x°° = — e*Q^. Summing up 

both sides of (fT9] l from i = 1 to we have 


^(1 + 2 ^ 1 ^ I ) - e*Q^^ 



which is equivalent to 

tv tv 

^[Qd] = + ‘^P\^^\y*iQd- 

i—1 2—1 

Here we use the fact that lies in the column space of 

L-, i.e., OL^ = L-b^ where € R^. Then = 

{L^b^yi = {b'^Y'{L'^1) = 0. Recalling that atavg = 
A n and \eiQ^ \ < f-, we finally obtain 


^[Qd] 


< 



A. 


The following example shows the tightness of this bound 
in this convergent case. Consider a simple two-node network 
with »"i = — I and r 2 = — Set both A and p to be 1. In 
this case, we have = 1, N = 2 and 


L_ = 


1 

-1 



We start with = — 1 and aj = ~^2 = 1- 

One can easily check that our initialization condition is met, 

^ilQd] = = 1, fc = 1, 2, • • •, 

in the updates of (11^ . Hence = — 1 and the consensus 
error is 


^[Qd] ~ 


avg 


1 2E 
2^ 


A. 


This coincides with the error bound in ( |2^ . 

Cyclic case: When the DQ-CADMM cycles with a period 
T, we must have = s^. Thus, for k > ko, we have that 


0 = a 


k+T 


- a'= = pL. Y. 


X 


fc+/ 

[Qd]’ 


/=1 


and consequently, ^[Qd] reaches a consensus, i.e., 

is true. Now denote 


1 _ ^ 

^[Qd] rj^ ^i[Qd] 


i = 1, 


We then get 




^[Qd] ^ 

Z=1 


fc+Z 


4 e 


^i[Qd] 


Z=1 


A 

< 2 - (29) 


T 


E 

z=i 


xY^ = 


1 + 2 p|A/i| 


ry |2p|A/i|a;[Q^] - t^'Y ' 


Finally, using (l29l l and following the same steps as in the 
convergent case we conclude that 


^IQd] 


< 


1 


p- 


2E\ 


N J 


A. 


Remark 1: The result that deterministic quantization may 
lead the consensus algorithm to either convergent or cyclic 
cases is also reported in im. Similar to theirs, one can use the 
history of agents’ variables, e.g., running average, to achieve 
asymptotic convergence at each node. Differently, while they 
can make local variable values close to the true average in 
cyclic cases without guaranteeing a consensus, our algorithm 
can reach a consensus but does not make the error arbitrarily 
small in general. 

Remark 2: We shall mention that or X[Qd] need 

not be unique. This is because, unlike the ideal CADMM, 
||tt^ — u*\\a in the DQ-CADMM need not decrease mono- 
tonically due to the quantization that occurs on at each 
update. Note also that practical consensus value does not 
necessarily meet the error bound and we usually have smaller 
errors than (|2^ in practice (see Fig.|2]). We hence expect better 
consensuses when (a;°,a°) are initialized closer to the ideal 
optima, which leads to a two-stage algorithm for quantized 
consensus in Section IVTl 

Remark 3: An interesting observation of our main result is 
the ADMM parameter p. While a small p indicates a small 
consensus error bound, the current paper does not quantify 
how it affects the convergence rate. Here we do not study the 
optimal selection of p but simply set p = 1. Therefore we do 
not regard p as a factor affecting our algorithm’s performance. 
We refer readers to ll22ll . Il28ll . ll^ for detailed discussions on 
how p affects the ADMM’s performance. 

Remark 4: Theorem 0] for rounding quantization extends 
straightforward to other deterministic quantizations as the only 
information used in our proof is the bounded quantization 
error. In contrast with H, d where the algorithms may 
fail for some deterministic quantization schemes, e.g., the 
rounding quantization, our results work for all deterministic 
quantization schemes as long as a finite quantization error 
bound is provided. 

Remark 5: In both the PQ-CADMM and DQ-CADMM 
iterations, x^ is quantized for the {k + l)th update at node 
i even though nodes can compute and store real values with 
infinite precision. The reason is to guarantee that lies 
in the column space of L- and thus the ideal CADMM 
update in either the PQ-CADMM or the DQ-CADMM [cf. 
Equation (l2Tl i1 possesses the linear convergence property given 
in Theorem |2l If we do not quantize x\ at its own node. 
Theorem |3] still holds due to j] = E[a;f] while Theorem 

|4]may fail. 

Remark 6: In the problem reformulation dUi, each node i 
has its local objective function being \{xi — rY and Ixavg 
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minimizes the global objective function f{x) which is the 
sum of the local objectives. To analyze the DQ-CADMM, 
we first identify the CADMM update in the matrix form as 
gk+i _ j-fgk Yvhere D is fixed throughout the iterations. 
We then write the DQ-CADMM update as the sum of the 
ideal CADMM update plus an accumulated error term and 
finally utilize the linear convergence rate of the CADMM [cf. 
Equations (l25l l and (I26l l1. In general, if the local objective 
functions do not have linear gradients or the linear convergence 
rate is not guaranteed (e.g., the LASSO is not differentiable 
and the corresponding CADMM update in this paper’s fashion 
does not converge linearly), then the current proof no longer 
holds with deterministic quantization. 

VI. ADMM Based Algorithm for Quantized 
Consensus 

Let us summarize the two quantized versions of the 
CADMM: the PQ-CADMM converges linearly to the data 
average in the mean sense, but it does not guarantee a 
consensus within finite iterations; the DQ-CADMM, on the 
other hand, either converges to a consensus or cycles with the 
same mean of quantized variable values over one period at 
each node after a finite-time iteration, but results in an error 
from the average. 

As discussed in Remark |2] we can first run the PQ- 
CADMM 2K times to obtain Xi = which 

is a reasonable estimate of at node i according to 
Theorem [3] Here K can be chosen such that W^xf^Q j] is 
close enough to ccavg when we have the knowledge of agents’ 
data and the network topology. Otherwise, we can simply 
set K = [low (logio(s + 1) + l) max{- log^o P, 1}] or as 
large as permitted. Note also that oii = Y^}^k+i is 

also a good estimate of a* = ri — Xavg, and that a = 
[ai; 0 : 2 ; • • • ; aw] = satisfies the initialization 

condition as cc*’ lies in the column space of T_. We can 
therefore run the DQ-CADMM with this Xi and cti as initial 
values. The probabilistically quantized CADMM followed by 
deterministically quantized CADMM (PQDQ-CADMM) is 
presented in Algorithm [T] 

VII. Simulations 

This section investigates the performance of the DQ- 
CADMM and the PQDQ-CADMM via numerical examples. 
Since existing methods with dithered quantization do not 
guarantee convergence to a consensus in finite iterations, we 
only compare our algorithms with those that uses deterministic 
quantization to reach a consensus, i.e., the gossip based 
method in ll 20 l and the classical method in ifTTI . 

A. Performance of the PQDQ-CADMM, the DQ-CADMM, the 
gossip based method, and the classical method 

To construct a connected graph with N nodes and E edges, 
we first generate a complete graph consisting of N nodes, 
and then randomly remove _ E edges while ensuring 

that the network stays connected. Set A = 1 and assume that 
agents’ data have very high variances in large networks, e.g., 
let ri ~ Our settings are 


Algorithm 1 PQDQ-CADMM for quantized consensus 
Require: Initialize = 0, q;° = 0, and p > 0. Set K = 
[low (logio(s + 1) + 1) max{- logig P, 1}] ■ 

1 : for A: = 0,1, • • • , 2K — 1, every node i do 
2 : 




\ feA/i 


- a,; + rz 


\ few / 

3: end for 

4: set ^ ^ EfE+i and 

k = 2K. 


5 : repeat 

6 : Lor i = 1,2, • • • , W, 



7: set k = k 1. 

8 : untU a predefined stopping criterion (e.g., a maximum 
iteration number) is satisfied. 



Fig. 1. Iterative en'or versus iterations where each plotted value is the average 
of 1000 runs. 
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5 lo'* 


- PQDQ-CAOMM I 

- DQ-CADMM I 

- Gossip based method | 

• Classical method f 


50 100 150 200 250 300 350 400 

Number of nodes 



(a) 


(b) 


(c) 


Fig. 2. Consensus en'or of the four algorithms where A = 1 and the plotted values are the average of 100 runs; (a) fixing Af = 50 and varying E E [49,1225], 
(b) fixing E = 400 and varying N £ [29,399], (c) fixing ^ = 10 and varying N G [20,200]. 





Fig. 3. Convergence time of the four algorithms where A = 1 and the plotted values are the average of 100 runs; (a) N = 50 and E G [49,1225], (b) 
E = 400 and N G [29, 399], (c) ^ = 10 and IV G [20, 200]. 


PQDQ-CADMM: Set p = 1. 

DQ-CADMM: Set p = 1, x° = 0 and a° = 0. 

Gossip based method: We randomly pick one edge in A 
and perform the updating, i.e., if (i,j) G is chosen, 

thenar = 

Classical method: Let W denote the weight matrix of the 
graph Qd = {V,A}. The updating rule is then given by 

a.fe+1 = 

where the subscript [q^^\ denotes the 
rounding down quantization. We utilize the Metropolis 
weights defined in 1 ^ : 


r(l+max{|M|, |A/'j|}) \ (z,j)GA 
I 0, otherwise. 


We simulate a connected network with N = 50 

nodes and E = 500 edges. Define the iterative error as 
IIicIqj — ItCavglb/V^ which is equal to the consensus error 
|a:*g] — atavgl when consensus is reached. Plotted in Fig. [T]is 
the Iterative error of the four algorithms at every iteration k 
with each value being the average of 1000 runs. Note that 
we start the plot of the PQDQ-CADMM from the {K -f l)th 
iteration as its first K iterations are used only to reach a 


neighborhood of ccavg; at the {2K -f l)th iteration, 
is updated based on the running average of the {K -[- l)th 
iteration to the 2A"th iteration. The figure indicates that all 
the four algorithms converge to a consensus at one of the 
quantization levels. The average consensus error of the DQ- 
CADMM is 1.21, which is much smaller than the upper bound 
(i-l-^)A = 20.5. One can also see that the PQDQ-CADMM 
converges almost immediately after the 2Kt\\ iteration. In the 
following we compare the consensus error and the convergence 
time of the four algorithms via simulations that respectively 
fix the number of nodes, the number of edges, and the average 
degree of the graph. 

Consensus error: In Fig. |2(a)| we fix A = 50 and vary 
E until the graph is complete. The gossip based method and 
the classical method have decreasing consensus errors as E 
increases. The consensus error of the DQ-CADMM, however, 
becomes larger as the average degree and therefore the error 
bound increase. The PQDQ-CADMM has the smallest con¬ 
sensus error whose average of 100 runs is less than 0.40 for 
all E. We then fix A = 400 and let N vary. Fig. |2(b)| shows 
that the gossip based method and the classical method have 
increasing consensus errors as N increases. The consensus 
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error of the DQ-CADMM, on the contrary, decreases when 
N becomes larger. The PQDQ-CADMM also has the smallest 
consensus error in this case. In the last setting we fix the 
average degree ^ = 10 while varying N. The classical 
method and the gossip based method then both have increasing 
consensus errors when N and thus the range of agents’ data 
increase. The consensus error of the DQ-CADMM is relatively 
small compared with the upper bound (0.5+^)A = 10.5 
and decreases when N becomes larger. The proposed PQDQ- 
CADMM still has the smallest consensus error whose average 
of 100 runs is less than 0.2 for all N. 

We conclude that the consensus error of the gossip based 
method and the classical method depends on the average 
degree of the graph as well as the range of agents’ data. 
Note that their consensus errors can be extremely large for a 
sparsely connected graph. The DQ-CADMM has an increasing 
consensus error when the average degree increases while the 
PQDQ-CADMM performs almost the same for all network 
structures in terms of the consensus error. 

Convergence time: We study the convergence time of the 
four algorithms via numerical examples in Fig. [3] Since the 
gossip based method involves only one edge and the other 
three methods utilize all the edges at each iteration, we plot 
also the quotient of the convergence time of the gossip based 
method divided by the number of edges, namely. Gossip based 
method adjusted, in the figure. 

In Fig. |3(a)| the gossip based method and the classical 
method converge slower as the graph becomes sparser. When 
the average degree is fixed, they have longer convergence time 
as N increases. Therefore, the convergence time of the gossip 
based method and the classical method is also affected by 
the average degree of the graph and the range of agents’ 
data. Different from the gossip based and classical methods, 
we see in Fig. |3(a)| that the convergence time of the DQ- 
CADMM increase as the graph becomes denser. In Fig. |3(b)| 
and Fig. |3(c)| however, the convergence time also increases 
while the graph becomes sparser, which is possibly because 
of the increased distance between starting points and optimal 
values. Exactly characterizing the convergence time of the DQ- 
CADMM is beyond the scope of the current paper and will be 
treated as future work. For the PQDQ-CADMM, we observe 
that the significant portion of its convergence time is spent on 
achieving an approximate estimate of Xavg, i e., running the 
PQ-CADMM with 2K iterations. With good starting points, 
the DQ-CADMM converges almost immediately. 

B. Performance of the PQDQ-CADMM with different quanti¬ 
zation resolutions 

We next consider the effect of the quantization resolution 
on the PQDQ-CADMM. Fig. |4] plots consensus errors of 
the PQDQ-CADMM with iV = 50 and £; G [49,1225] 
for A G {0.02,0.1,0.5,2.5}. The consensus error tends to 
increase on the average as the quantization resolution becomes 
larger, which is not surprising since a coarse quantization 
indicates a higher loss of information at each update. We then 
calculate the ratio of the consensus error to the quantization 
resolution: the plotted values, which are the averages of 


100 runs, all lie in (0.227A, 0.337A) and the variances are 
less than 0.051. Moreover, the convergence time of each 
quantization resolution has a mean of {2K-\-2.1) iterations and 
a variance less than 0.0008, which coincides with our previous 
analysis that the PQDQ-CADMM converges immediately after 
the first 2K iterations. 



0.02 


.|Q-3 -1-1-1-1-1-iJ 

200 400 600 800 1000 1200 

Numer of edges 

Fig. 4. Consensus error of the PQDQ-CADMM with different quantization 
resolutions, i.e., A G {0.02, 0.1, 0.5, 2.5}, for N = bO and E G [49,1225]; 
each plotted value is the average of 100 runs. 


C. Cyclic Cases 

While we prove that the DQ-CADMM either converges 
or cycles in Theorem 4, it is noted that the above numeri¬ 
cal examples all lead to reach convergence results. Indeed, 
the proposed deterministic algorithms, the DQ-CADMM and 
PQDQ-CADMM, converges in most cases as shown by the 
following simulation. For connected networks with N nodes, 
we consider star graph which has the smallest average degree, 
randomly generated graph that has intermediate average de¬ 
gree, and complete graph that has the largest average degree. 
The result is given in Fig. |5] where the y-axis represents the 
number of cyclic cases in 10^ trials. Clearly, the DQ-CADMM 
and PQDQ-CADMM with fixed parameter p = 1 converge in 
most cases, particularly with large networks. 

VIII. Conclusion 

In this paper we have proposed an efficient algorithm, the 
PQDQ-CADMM, for quantized consensus problems. We first 
study the effects of both probabilistic and deterministic quan¬ 
tizations on the CADMM. With probabilistic quantization, the 
PQ-CADMM converges linearly to the data average in the 
mean sense. In the deterministic case, we can bound the sum 
of the absolute value of each error term caused by quantization 
using the global and linear convergence of the CADMM 
and thus prove that the DQ-CADMM either converges or 
cycles. We finally combine the two quantized versions of the 
CADMM to obtain the PQDQ-CADMM algorithm, where the 
PQ-CADMM to is used to get an initial estimate of the data 














12 



0 50 100 150 200 250 300 

Number of nodes 


Fig. 5. Number of cyclic cases in 10^ trials. 

average and the DQ-CADMM is used subsequently for con¬ 
sensus reaching purpose. Simulations show that our PQDQ- 
CADMM provides the best result than all existing methods 
using deterministic quantization in terms of the consensus 
error. 

Our approach also motivates a number of further research 
directions; 

1) Data communications between agents were assumed to 
be perfect in this paper. In practice, channel impairment 
may lead to imperfect transmissions. Moreover, the links 
between agents may fail and the network topology may 
vary randomly, as studied in ifTTI . ifTSll . ifTSll . It is thus 
meaningful to investigate how our algorithm performs in 
these settings. 

2) The algorithm parameter p is another interesting topic 
in the DQ-CADMM. Roughly speaking, a smaller p 
may result in a small consensus error but a longer time 
to reach the convergent or cyclic result. Therefore, tts 
choice should be guided depending on whether a small 
consensus error or fast consensus speed is desired. 

3) We only considered the unbounded quantization scheme 
in this paper. It is also interesting to consider bounded 
quantization that is used in many applications as it 
significantly reduces the amount of data that needs to 
be exchanged. 

Appendix 

Proof of Lemma We first manipulate ® to derive 
equivalent updates 

Vf{x^+^) -f -f pA^B{z'^ - ^'=+1) = 0, (30) 

^Tyk+l ^ 

- p[Ax^+^+Bz^+^) = Q, (32) 

where (l30l l and OTl) are from multiplying the two sides of the 
A-update by and B^ and adding them to the tc-update and 


2 ;-update, respectively. Recalling A = [/ 3 ; 7 ] with / 3,7 £ 
and B = [—I 2 E', —I 2 E], we know that = — 7^+1 from 

OTl) . Since we initialize /3° = — 7 °, we have B^X^ — 0 
for /c = 0 , 1 , • • •. Equation Ool) then reduces to V/(£C^+^) -|- 
M-(3^'^^—pM+{z^ — z^'^^) = 0 , and 02 l) splits into (3^^^ — 
j3^ — pAiX^^^ + pz^'^^ = 0 and 7 ^^+! — jk _ pA 2 X^^^ -f 
pz^+i = Q Summing and subtracting these two equations we 
have — z^^^ = 0 and — = 

0. With the initialization 2 :° = ^M’fx^ — z^ — Q 

holds true for k = 0,1, • • •. Since x* is unique and equal to 
ItCavg according to Lemma [T] 2 ;* = ^Mf^x* is also unique. 
To summarize, with the initialization /3° = — 7 ° and 2 ° = 
iATjtc®, O0l i- (l32l i reduce to 

V/(£c'=+^) + - pM+{z^ - 2 '^+^) = 0, (33) 

^fe+i _ _ ^mTx'^+^ = 0, (34) 

- z^+^ = 0, (35) 

which further lead to x^ ^ x* = ItCavg and 2 ^ — ^M^x* = 
^Mjliavg uniquely as fc — 00 . Taking A: —>• 00 in OTt-OSl) 
and using global convergence, we get 

Vfix*) + M_/3* = 0, (36) 

M^x* = 0, (37) 

^M^x* - z* =0. (38) 

We can now use (l36l l to demonstrate the uniqueness of /3* if 
we also initialize /3o in the column space of Mf. Note that 
if /3° lies in the column space of Mf then OTl i indicates that 
/3^ also lies in the column space of Mf, fc = 0,1, • • •. The 
uniqueness of /3* then follows from the uniqueness of x* and 
Lemma 

Next we show the linear convergence of . Subtracting 
(El-dS]) from (ITbl l- dTsT l. respectively, and using Vfix) = 
X — r, we have 

x’^+^ -X* = pM'liz’^ - z^+^) - M„(/3'=+^ -(3*), (39) 
-X*) = [3'^+^ - f3'^, (40) 

-X*) = z^+'^ - z*. (41) 

We therefore obtain 

Wx’^+^-xX 

(a;fc+i _ x*,pMl{z'^ - 2 '=+!) - - (3*)) 

= 2p{z^ - 2 ^=+^ 2 ^=+^ - z*) + -{I3^- - (3*) 

P 

(J _ u*\\l - (42) 

where (a) is from ( 1391 ), ( 6 ) is from dTOl i and dTTT i. and (c) is 
from the definitions of u and G. Due to (02]), to prove (ITTl ) 
we only need to show 

ll«^' - - *111 > 
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which is equivalent to 

p||^fc+l _ + ipfc+1 _ ^/c||2 ^ ii^fc+1 _ ^*||2 

It then suffices to show 


p\\z^+^-z'^M + \\x^+^ -X 


„*l |2 


> Sp\\z^+^ - z*lll + -11/3'=+! - (3*\\l (43) 

P 

The rest of this proof is to establish that 5p\\z^'^^ — z*jl 2 and 
^11/3^+^ — /3*||| are upper bounded by two non-overlapping 
parts of the left side of ( l43T l, respectively. 

We first have from (l4Tl i that 




2 = -IIMi(x^+^-x 


*M|2 
2 


„*l |2 


< l^,a^AM+)\\x + - a;*||2. 


(44) 


To upper bound ||/3^'+^ — /3*|||, we first notice that /3^'+^ — (3* 
lies in the column space of M."£. Therefore, 

||M_(/3'=+i - /3*)||2 > a^i„(M_)||/3'=+i - f3*\\l (45) 

Now using (l45l l and (|39] | we get 


< 


11/3"+'-ni2 

1 




[X 


k+1 


— X*) — pMj^{z^ — z 


fe+l ^||2 


(a) 

< 


„fc+l 


2 


-x*\\i + p^\\M+{z^-z 


fc-l-lM| 2 t 






X ' — X 


*l |2 


+ P^o-Lx(-^+)ll^"+' - (46) 

where (a) is from the Cauchy-Schwarz inequality together 
with the fact 2 pip 2 < Pi + pI for ^riy pi, P 2 G R. Combining 
(l44l i and (l46T l. we have 


P- 

+ 


2CT^ax(3W+)| 


'fiin(M-) 
1 




rP^max 


(M+) 


" 11 ^ 

2 




- X 

^*||2 1 


*l|2 
2 


>p||^'=+^-^*||^ + -||/3'“'+^-/3*||^. 
P 


The proof is thus complete by picking 


S = min 


2a2_(M+) ’ + 8 j ’ 
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